With a photograph grouping device of this kind, photographs are managed by grouping on a photographing time and date basis or when a time interval between successively taken photographs exceeds a fixed threshold value, by putting photographs into different groups (e.g. Patent Literature 1).
FIG. 19 is a block diagram showing an example of a structure of a photograph grouping device which groups photographs by subjecting a time interval of photographing times to threshold processing as recited in Patent Literature 1. The photograph grouping device shown in FIG. 19 comprises an adjacent photographs block photographing time interval calculating unit 901 and a time interval threshold processing unit 902. With time series photographing time information indicative of photographing time of each photograph taken in time series as an input, the adjacent photographs block photographing time interval calculating unit 901 transfers time series photographing time interval information indicative of a photographing time interval between respective photographs aligned in time series to the time interval threshold processing unit 902. With the time series photographing time interval information output from the adjacent photographs block photographing time interval calculating unit 901 as an input, the time interval threshold processing unit 902 outputs group boundary position information indicative of a position of a group boundary in a group of photographs to be grouped.
Next, operation of the photograph grouping device shown in FIG. 19 will be described. To the adjacent photographs block photographing time interval calculating unit 901, time series photographing time information as photographing time data of photographs taken in time series which is extracted from photographs or related information attached thereto is input in the order of photographing time. Since it is a common practice that photographing time data is stored within a photograph in the Exif format, reading the information obtains time series photographing time information.
The adjacent photographs block photographing time interval calculating unit 901 calculates a difference between photographing times of photographs adjacent to each other in time from the applied time series photographing time information. More specifically, with the total number of photographs input as N and photographing time of an i-th photograph as T(i), execute the following Expression (1) with respect to each i (i=1, . . . , N−1) to output an obtained time interval d(i) (i=1, . . . , N−1) as time series photographing time interval information to the time interval threshold processing unit 902.d(i)=T(i+1)−T(i)  Expression (1)
The time interval threshold processing unit 902 compares the applied time series photographing time interval information with a fixed threshold value and when the time interval exceeds the threshold value, identifies its position as a group boundary. More specifically, with the threshold value as TH, execute the following Expression (2) with respect to each time interval d(i) (i=1, . . . , N−1) to identify i satisfying the Expression (2) as a group boundary, that is, determine that a break-point of the group exists between the i-th photograph and the (i+1)th photograph. The time interval threshold processing unit 902 obtains all i satisfying the Expression (2) and outputs the same as group boundary information.d(i)>TH  Expression (2)
While in the above-described example, the threshold value TH for use in dividing an event is fixed, disclosed in Non-Patent Literature 1 is the method of dynamically changing a value of a threshold according to a fixed number of preceding and succeeding photographing time intervals to group photographs.
FIG. 20 is a block diagram showing an example of a structure of a photograph grouping device which groups photographs by dynamically changing a threshold value to subject a time interval of a photographing time to the threshold processing as recited in Non-Patent Literature 1. The photograph grouping device shown in FIG. 20 comprises the adjacent photographs block photographing time interval calculating unit 901, the time interval threshold processing unit 902, a fixed number of photographing time interval data selecting unit 911 and a threshold value determining unit 912.
In the example shown in FIG. 20, with time series photographing time information as an input, the adjacent photographs block photographing time interval calculating unit 901 outputs time series photographing time interval information to the fixed number of photographing time interval data selecting unit 911 and the time interval threshold processing unit 902. With the time series photographing time interval information output from the adjacent photographs block photographing time interval calculating unit 901 as an input, the fixed number of photographing time interval data selecting unit 911 outputs a fixed number of pieces of photographing time interval data to the threshold value determining unit 912. With the fixed number of pieces of photographing time interval data output from the fixed number of photographing time interval data selecting unit 911 as an input, the threshold value determining unit 912 outputs a threshold value to the time interval threshold processing unit 902. With the time series photographing time interval information output from the adjacent photographs block photographing time interval calculating unit 901 and the threshold value output from the threshold value determining unit 912 as inputs, the time interval threshold processing unit 902 outputs group boundary position information.
Next, operation of the photograph grouping device shown in FIG. 20 will be described. When time series photographing time information is input, the adjacent photographs block photographing time interval calculating unit 901 calculates a difference between photographing times of photographs adjacent to each other in time to output time series photographing time interval information similarly to the example shown in FIG. 19.
The fixed number of photographing time interval data selecting unit 911 selects a fixed number of photographing time intervals preceding to and succeeding a time interval to be processed. More specifically, when a k-th time interval d(k) is to be processed, with a fixed number as w, select the number w of preceding and the number w of succeeding time intervals d(k−w), d(k−w+1), d(k), d(k+1), d(k+w). The selected time intervals are output as the fixed number of pieces of photographing time interval data to the threshold value determining unit 912.
The threshold value determining unit 912 calculates a threshold value TH(k) for use in the threshold processing of d(k) to be processed from the fixed number of pieces of photographing time interval data. More specifically, calculation will be made by using the following Expression (3). Here, K is a constant, and experimentally K is set to be log(17). The obtained threshold value TH(k) is output to the time interval threshold processing unit 902.
                    (                  FORMULA          ⁢                                          ⁢          1                )                                                                      TH          ⁡                      (            k            )                          =                  exp          (                      K            +                                          1                                                      2                    ⁢                                                                                  ⁢                    w                                    +                  1                                            ⁢                                                ∑                                      i                    =                                          -                      w                                                        w                                ⁢                                  log                  ⁡                                      (                                          d                      ⁡                                              (                                                  k                          +                          i                                                )                                                              )                                                                                )                                    Expression        ⁡                  (          3          )                    
Operation of the time interval threshold processing unit 902 is the same as that of the example shown in FIG. 19. Threshold value used is not fixed but changed according to each (i=1, . . . , N−1). In the threshold processing of d(k), for example, the threshold value TH(k) output from the threshold value determining unit 912 is used.    Patent Literature 1: Japanese Patent Laying-Open No. 2004-355493.    Non-Patent Literature 2: J. C. Platt, M. Czerwinski, B. A. Field, “Photo TOC: Automatic Clustering for Browsing Personal Photographs”, Proceedings of the 2003 Joint Conference on International Conference on Information, Communication and Signal Processing and Pacific Rim Conference on Multimedia, 2003, Vol. 1, pp. 6-10.
Use of the photograph grouping devices shown in FIG. 19 and FIG. 20 enables time series photographs to be grouped. There exists, however, a case where a precision in grouping (division precision) might be degraded. Among factors of degradation in division precision are as follows.
The first problem is that such division by a fixed threshold value as shown in FIG. 19 makes it extremely difficult to determine a threshold value which realizes highly precise grouping. The reason is that a fixed threshold value fails to sufficiently reflect a change of a character of an object or an event to be photographed and a user's photographing disposition. When a threshold value is determined based on an event in which photographs are sparsely taken, for example, there occurs a problem that finding a group boundary from a group of photographs of an event in which photographs are frequently taken will have a difficulty. Conversely, when a threshold value is determined based on an event in which photographs are frequently taken, for example, there occurs a problem that a group of photographs of an event in which photographs are sparsely taken will be excessively divided. In addition, since the frequency of photographing largely depends on a photographer's photographing disposition, it is highly possible that a threshold value determined reflecting a disposition of a specific person will not be appropriate for grouping photographs taken by others.
The second problem is that by such division by a threshold controlling method as shown in FIG. 20, when photographing times of photographs largely apart in time are included in a fixed number of pieces of photographing time interval data for use in determining a threshold value, the threshold value cannot be determined satisfactorily. The reason is that in a case of a photograph taken sparsely in time, a photographing time interval from a photograph largely apart in time might be included in the fixed number of pieces of photographing time interval data, so that a threshold value will be increased due to the effect of the largely apart photographing time interval, thereby degrading a division precision. Although photographs largely apart in time are fundamentally not related with the contents of photographs to be grouped at present, when the photographs are within a range of a number (number of photographs) as a selection reference, it will nonetheless affect determination of a threshold value. If the number of photographs for use in determining a threshold value is reduced in order to avoid such a situation, because in the grouping of photographs of an event whose photographing frequency is high, only the proximate photographs will affect determination of a threshold value, a threshold value might be unstable.
Under these circumstances, an object of the present invention is to provide a photograph grouping device, a photograph grouping method and a photograph grouping program which enable photographs to be grouped with high precision even when a character of an event to be photographed or a user's photographing disposition varies.